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Objective Lightning Prob- 
ability Tool, Phase III 

Peak Wind Tool for General 
Forecasting, Phase II 

ADAS Update and Maintain- 
ability 

Verify MesoNAM Perform- 
ance 


This Quarter’s Highlights 


Launch Support 


Dr. Merceret and Ms. Craw- 
ford supported the Atlas V 
launch on 14 August. 


Three AMU tasks were completed in this Quarter, each resulting in a 
forecast tool now being used in operations and a final report docu- 
menting how the work was done: 

• Mr. Barrett completed Phase II of the Peak Wind Tool for General Forecasting task by delivering an 
improved wind forecasting tool to operations and providing training on its use; 

• Dr. Watson completed a graphical user interface (GUI) she updated with new scripts to complete 
the ADAS Update and Maintainability task, and delivered the scripts to the Spaceflight Meteorology 
Group on Johnson Space Center, Texas and National Weather Service in Melbourne, Fla.; and 

• Dr. Bauman completed the Verify MesoNAM Performance task after he created and delivered a 
GUI that forecasters will use to determine the performance of the operational MesoNAM weather 
model forecast. 
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Quarterly Task Summaries 

This section contains summarizes of the AMU activities for the fourth quarter of Fiscal Year 2010 (July - 
September 2010). The accomplishments on each task are described in more detail in the body of the report 
starting on the page number next to the task name. 


Objective Lightning Probability Tool, 
Phase III ( Page 4 ) 

Purpose: Re-create the lightning probability forecast equations 
used in 45th Weather Squadron (45 WS Operations with new 
data and stratifications based on the progression of the lightning 
season. These modifications were anticipated to improve the per- 
formance of the equations used to make the daily lightning prob- 
ability forecasts for operations on Kennedy Space Center (KSC) 
and Cape Canaveral Air Force Station (CCAFS). 

Accomplished : New equations were created for each sub- 
season and their performance was tested against the Phase II 
equations in current operational use. The new equations outper- 
formed several forecast methods, but caused a degradation in 
the forecast by as much as 12% compared to the Phase II equa- 
tions. Therefore, the Phase III equations will not be transitioned 
to operations. The superior Phase II equations will remain in the 
operational objective lightning probability tool. 
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Peak Wind Tool for General Forecasting, 

Phase II ( Page 6 ) 

Purpose: Update the 45 WS tool, developed by the AMU in 
Phase I, that forecasts the peak wind speed during the cool 
season (October-April). This tool forecasts the peak wind 
speed for the day from any of the towers on KSC/CCAFS and 
its associated mean speed, and provides the probability of 
issuing wind warnings in the KSC/CCAFS area. The period of 
record was expanded to increase the size of the data set 
used to create the forecast equations and new predictors 
were evaluated. 

Accomplished: The Phase I and Phase II tools were com- 
pared, and the Phase II tool had superior performance. This 
tool was delivered and a training session on its use was con- 
ducted for the 45 WS. The final report was completed after 
making modifications suggested in the internal AMU and ex- 
ternal customer reviews. It is now available on the AMU web- 
site: http://science.ksc.nasa.gov/amu 
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Quarterly Task Summaries 





Verify MesoNAM Performance 

( Page 6 ) 

Purpose: Verify the performance of the 12-km North 
American Mesoscale model (MesoNAM) forecasts for 
CCAFS and KSC. The verification consisted of an objec- 
tive statistical analysis comparing the MesoNAM forecast 
winds, temperature and moisture, and their changes over 
time, to the observed values at customer-specified KSC / 
CCAFS wind towers. This objective analysis and the re- 
sulting GUI helps forecasters understand the model’s 
strengths and weaknesses, resulting in improved fore- 
casts for operations. 

Accomplished: The charts in the GUI were updated with 
data from additional sensor heights and delivered to the 
45 WS in July. The final report was completed after inter- 
nal AMU and customer reviews and was delivered in 
September. It is available on the AMU website: 
http://science.ksc.nasa.gov/amu 




ADAS Update and Maintainability 
( Page 7 ) 

Purpose: Acquire the latest version of the Advanced 
Regional Prediction System (ARPS) Data Analysis 
System (ADAS) for the local data integration system 
(LDIS) at the National Weather Service in Melbourne, 
Fla. (NWS MLB) and the Spaceflight Meteorology 
Group (SMG) at Johnson Space Center, Texas. Up- 
date the AMU-developed shell scripts to govern the 
LDIS so it can be easily maintained, and update the 
ADAS GUI. 

Accomplished: The analysis of the error statistics 
for the MADIS data showed that altering the error sta- 
tistics for each data source had little impact on the 
ADAS analyses. Therefore, the error values used in 
the previous version of the LDIS scripts were not 
changed. The updated scripts were installed at NWS 
MLB and SMG. Installation instructions and a user’s 
guide are included in the final report. 
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AMU ACCOMPLISHMENTS DURING THE PAST QUARTER 

The progress being made in each task is provided in this section, organized by topic, 
with the primary AMU point of contact given at the end of the task discussion. 

SHORT-TERM FORECAST IMPROVMENT 


Objective Lightning Probability 
Tool, Phase III (Ms. Crawford) 

The 45th Weather Squadron (45 WS) includes the prob- 
ability of lightning occurrence in their daily morning brief- 
ings. This information is used by forecasters when de- 
termining the likelihood of violating launch commit crite- 
ria and weather flight rules, and planning for daily 
ground operations on Kennedy Space Center (KSC) 
and Cape Canaveral Air Force Station (CCAFS). In 
Phase I, the AMU developed a set of equations that cal- 
culate the probability of lightning occurrence for the day 
(Lambert and Wheeler 2005) and 
a graphical user interface (GUI) to 
display the output. These equa- 
tions outperformed several fore- 
cast methods used in operations. 

The GUI allowed forecasters to 
interface with the equations by en- 
tering predictor values to output a 
probability of lightning occurrence. 

In Phase II (Lambert 2007), two 
warm seasons were added to the 
period of record (POR), the equa- 
tions redeveloped with the new 
data, and the GUI transitioned to 
the Meteorological Interactive Data 
Display System (MIDDS). The 
MIDDS GUI retrieves the required 
predictor values automatically, re- 
ducing the possibility of human error. In this phase, 
three warm seasons (May-September) will be added to 
the POR, increasing it to 20 years (1989-2008), and 
data for October will be included. The goal of this phase 
is to create equations based on the progression of the 
lightning season instead of creating an equation for 
each month. These equations will capture the physical 
attributes that contribute to thunderstorm formation 
more so than a date on a calendar. 

Sub-Season Start Dates 

None of the three methods developed and tested by 
Ms. Crawford, described in the previous AMU Quarterly 
Report (Q3 FY10), were able to discern the sub-season 
start dates in each year. Ms. Crawford and Mr. Roeder 
agreed to end testing and, instead, define the dates us- 


ing the daily lightning climatology. The black Xs in 

Figure 1 show the beginning of each sub-season: 

• Ramp-up begins 18 May when the lightning fre- 
quency begins to increase; 

• Lightning begins 6 June when the rate of increase in 
the frequencies starts to decrease; 

• Ramp-down begins 17 August when the large de- 
crease in lightning frequency begins; and 

• Post begins 12 October when the rate of decrease 
lessens and the value reaches 0.13, the same as in 
the pre-lightning sub-season. 


Equation Development 

Ms. Crawford stratified the data by sub-season first, 
then into development and verification datasets. The 
amount of data available for equation development was 
critical to the reliability of the new equations. Ms. Craw- 
ford determined the number of records in the develop- 
ment datasets for each sub-season stratification, and 
found they met and exceeded the threshold of the 250 
events deemed necessary by the World Meteorological 
Organization (1992) in order to derive stable statistical 
relationships. Once satisfied that there were sufficient 
data, Ms. Crawford developed a set of five equations, 
one for each sub-season. 

As in Phases I and II, Ms. Crawford used the logistic 
regression method to create the five equations. She 
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Sub-Season Start Dates 



Figure 1. The 1989-2008 daily lightning climatology with the sub-season start 
dates indicated by black Xs. 
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conducted predictor selection for each individual sub- 
season to account for the possibility that different vari- 
ables may be more critical to convection formation as 
the lightning season progresses. Detailed descriptions 
of logistic regression and the predictor selection proce- 
dure are found in the Phase II final report (Lambert 
2007). 

Ms. Crawford developed and tested several versions 
of each equation, each with varying numbers of predic- 
tors. The version that performed best on the verification 
data set was chosen as the final equation. Table 1 
shows the predictors for each of the sub-season equa- 
tions in rank order of their importance in predicting light- 
ning. The predictor names are color-coded to highlight 
their occurrence in each equation. The first predictor in 
the first four equations, Thompson Index, accounts for 
instability and moisture in the profile, which are both 
necessary ingredients for thunderstorm formation. The 
flow regime probability accounts for the lifting mecha- 
nism, or lack thereof, from the low-level flow interacting 
with the sea breeze, which occurs almost daily in the 
warm season. 


Table 1. The predictors for each sub-season equation, in order of their im- 
portance in predicting lightning occurrence and colorized to highlight their oc- 
currence in each equation. 

Pre-Lightning 

Ramp-Up 

Lightning 

Ramp-Down 

Post-Lightning 

Thompson Index 

Thompson Index 

Flow Regime 

Persistence 

Thompson Index 

Flow Regime 

Persistence 

Thompson Index 

Flow Regime 
Vertical Totals 

Flow Regime 
Lifted Index 


Equation Testing 

Ms. Crawford used the predictors from the verifica- 
tion dataset in the Phase II and III equations to produce 
‘forecast’ probabilities for the tests. Using the verification 
dataset provided an assessment of equation perform- 
ance that could be used to conclude how the equations 
will perform in future operations. She compared the 
forecast methods using the Brier Skill Score (Wilks 
2006), which is a measure of equation performance ver- 
sus other forecast methods. In this case, it would be 
Phase III performance versus 1-day persistence and the 
daily climatological, flow regime, sub-seasonal, and 
Phase II probabilities. 

In Phase II, an equation was developed for each 
month, May-September. In order to 
conduct a fair comparison, Ms. Craw- 
ford stratified the verification data by 
month for the calculations, and used 
the Phase II flow regime values. After 
she calculated the probabilities for 
each month, she re-stratified the data 
into sub-seasons for comparison with 
the Phase III probabilities. 


The Phase III probabilities were calculated for all 
sub-seasons, May-October. Ms. Crawford could not 
compare the performance of the post-lightning sub- 
season equation to the Phase II equations since there 
was not an October equation from that work. The ramp- 
down season was compared, but only using data 
through the end of September. This caused 39 days out 
of 197 (~20%) in the ramp-up verification dataset to be 
excluded from the comparison. 

The Brier Skill Scores for each of the Phase III equa- 
tions and a composite result for the entire warm season 
are shown in Table 2. The Phase III equations show 7- 
57% improvement in skill over the first four methods in 
the table. However, their performance against the 
Phase II equations was poor. In no sub-season did the 
Phase III equations outperform the Phase II equations. 

The degradation in skill of the Phase III equations 
could have several causes. The development datasets 
for the pre-lightning and ramp-up seasons had fewer 
samples than the monthly datasets in Phase II. More 
cases may result in better predictor selection. However, 
the lightning and ramp-down sub- 
seasons had more samples in 
their development datasets and 
the equations were still under- 
performers. The data were not 
stratified by sub-season in each 
individual year, but the same start 
dates were used in every year. It 
is likely there was overlap of sub- 
season days at the beginning and 
end of each sub-season in each individual year, which 
could affect equation performance. Regardless of the 
cause, the Phase III equations produced a degradation 
in skill and will not be transitioned to operations. 

Final Report 

Ms Crawford wrote an initial draft of the final report 
and submitted it for internal AMU review. Once an exter- 
nal customer review is completed and Ms. Crawford 
makes the appropriate modifications to the report, it will 
be distributed to the customers. She will post it on the 
AMU website when she receives NASA approval. 

Contact Ms Crawford at 321-853-8130 or 

crawford.winnie@ensco.com for more information. 


Table 2. Percent (%) improvement or degradation (red ) in skill of Phase 
III over Phase II and other standard forecast methods. 

Forecast Method 

Pre-Ltg 

Ramp-Up 

Ltg 

Ramp-Dn 

Post-Ltg 

Season 

Persistence 

52 

48 

51 

47 

57 

50 

Daily Climatology 

17 

18 

25 

23 

23 

23 

Sub-Season Climatology 

18 

22 

25 

27 

21 

31 

Flow Regime 

7 

13 

7 

15 

18 

11 

Phase II Equations 

-12 

-12 

-0.6 

-4.1 



-3.6 
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Peak Wind Tool for General Fore- 
casting, Phase II (Mr. Barrett) 

The expected peak wind speed for the day is an im- 
portant element in the daily morning forecast for ground 
and space launch operations at KSC and CCAFS. The 
45 WS must issue forecast advisories for KSC/CCAFS 
when they expect peak gusts to exceed 25, 35, and 50 
kt thresholds at any level from the surface to 300 ft. In 
Phase I of this task (Barrett and Short 2008), the AMU 
developed a tool to help forecast the highest peak non- 
convective wind speed, the timing of the peak speed, 
and the average wind speed at the time of the peak 
wind from the surface to 300 ft on KSC/CCAFS for the 
cool season (October - April). For Phase II, the 45 WS 
requested that additional observations be used in the 
creation of the forecast equations by expanding the 
POR. In Phase I, the data set included observations 
from October 2002 to February 2007. In Phase II, obser- 
vations from March and April 2007 and October 2007 to 
April 2008 were added. To increase the size of the data 
set even further, the AMU added data prior to October 
2002. Additional predictors were evaluated, including 
wind speeds between 500 ft and 3000 ft, static stability 


classification, Bulk Richardson Number, mixing depth, 
vertical wind shear, inversion strength and depth, wind 
direction, synoptic weather pattern and precipitation. 
Using an independent data set, the AMU compared the 
performance of the Phase I and II tools for peak wind 
speed forecasts. The Phase II equations had better per- 
formance and were transitioned to operations. 

As in Phase I, the tool was delivered as a Microsoft 
Excel GUI. In addition, at the request of the 45 WS, the 
AMU made the tool available in MIDDS, their main 
weather display system. This allows the tool to ingest 
observational and model data automatically and pro- 
duce 5-day forecasts quickly. 

Final Report and Training 

Mr. Barrett provided training on the Peak Wind Tool 
to the 45 WS. covering how to use the Excel and 
MIDDS versions of the tool. He also completed the first 
draft of the final report and modified it based on recom- 
mendations received from internal AMU and external 
customer reviews. He then distributed the report to the 
customers and submitted a request to NASA for public 
release of the report. 

Contact Mr. Barrett at barrett.ioe@ensco.com 


MESOSCALE MODELING 


Verify MesoNAM Performance 
(Dr. Bauman) 

The 45 WS launch weather officers use the 12-km 
NAM (MesoNAM) text and graphical product forecasts 
extensively to support launch weather operations. How- 
ever, the actual performance of the model has not been 
measured objectively. In order to have tangible evi- 
dence of model performance, the 45 WS tasked the 
AMU to conduct a detailed statistical analysis of model 
output compared to observed values. The model prod- 
ucts are provided to the 45 WS 
by ACTA, Inc. and include 
hourly forecasts from 0 to 84 
hours based on model initializa- 
tion times of 00, 06, 12 and 18 
UTC. The objective analysis will 
compare the MesoNAM fore- 
cast winds, temperature and 
dew point, as well as the 
changes in these parameters 
over time, to the observed val- 
ues from the sensors in the 
KSC/CCAFS wind tower net- 
work shown in Table 3. Objec- 
tive statistics will give the fore- 
casters knowledge of the 
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model’s strength and weaknesses, which will result in 
improved forecasts for operations. 

GUI Update and Final Report 

Dr. Bauman updated the GUI to include data from 
the additional sensor heights shown in the far right col- 
umn of Table 3. He delivered the updated GUI to the 45 
WS in July. The final report was completed in Septem- 
ber and uploaded to the AMU web site. 

For more information contact Dr. Bauman at 321 - 
853-8202 or bauman.bill@ensco.com 


Table 3. Towers, launch activities and sensor heights at KSC and CCAFS 
used in the objective analysis to verify the MesoNAM forecasts. Additional sen- 
sor heights were added last quarter as shown in the right hand column. 


Tower Number 


Supported Activity and Facility 


Original Sensor 
Heights (ft) 


Additional Sensor 
Heights (ft) 


002 


Delta II (LC-17) 


6, 54, 90 


145, 204 


006 


Delta IV (LC-37) / Falcon 9 (LC-40) 


54 


6, 12, 162, 204 


0108 


Delta IV (LC-40) / Falcon 9 (LC-40) 


54 


6 , 12 


0110 


Atlas V (LC-41) / Falcon 9 (LC-40) 


54, 162, 204 


6 , 12 


0041 


Atlas V (LC-41) 


230 


393 / 394 


Shuttle / Constellation (LC-39A) 


60 


397 / 398 


Shuttle / Constellation (LC-39B) 


60 


511 / 512 / 513 


Shuttle Landing Facility 


6, 30 


y 



ADAS Update and Maintainability 
(Dr. Watson) 

Both the National Weather Service in Melbourne, 

Fla. (NWS MLB) and the Spaceflight Meteorology Group 
(SMG) have used a local data integration system (LDIS) 
since 2000 and routinely benefit from the frequent 
analyses. The LDIS uses the Advanced Regional Pre- 
diction System (ARPS) Data Analysis System (ADAS) 
package as its core, which integrates a wide variety of 
national and local-scale observational data. The LDIS 
provides accurate depictions of the current local envi- 
ronment that help with short-term hazardous weather 
applications and aid in initializing the local Weather Re- 
search and Forecasting (WRF) model. However, over 
the years the LDIS has become problematic to maintain 
since it depends on AMU-developed shell scripts that 
were written for an earlier version of the ADAS software. 
The goal of this task is to update the NWS MLB/SMG 
LDIS with the latest version of ADAS and upgrade and 
modify the AMU-developed shell scripts written to gov- 
ern the system. In addition, the previously developed 
ADAS GUI will be updated. 

Optimize Error Statistics 

The quality of the ADAS analyses is affected by user 
-configurable error parameters. Large (small) errors as- 
signed to a data source result in a smaller (larger) influ- 
ence of that data on the nearby grid points. Dr. Watson 
examined a set of control error parameters assigned to 
the observations, and then varied them to determine 
how much the background field was modified by the ob- 
servations with a different set of error parameters. She 
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did this to find out if the ADAS analyses could be im- 
proved by optimizing the error parameters for the Mete- 
orological Assimilation Data Ingest System (MADIS) 
mesonet data. 

The first test involved reducing the observational 
error variances to half of their control values while the 
second doubled the error variances. The results from 
both tests revealed that varying the error parameters for 
each data source had little impact on the ADAS analy- 
ses. The primary change that took place was an in- 
crease (decrease) in magnitude of some of the observa- 
tional “bull’s eyes” due to the greater (lesser) weight as- 
signed to the observations. In addition, multiple data 
sets available from MADIS did not have enough hourly 
observations available to fully optimize the error pa- 
rameters. Therefore, Dr. Watson left the values of error 
used in the previous version of the LDIS scripts unal- 
tered, and she created new error parameter files for the 
MADIS data will use the same error statistics. 

Site Visits 

Dr. Watson visited both NWS MLB and SMG to in- 
stall and configure the LDIS scripts on their local Linux 
systems. She included installation instructions and a 
user’s guide in the final report. 

Final Report 

Dr. Watson completed the final report after making 
modifications from internal AMU and external customer 
reviews. It will be distributed and posted on the AMU 
website when she receives NASA approval. 

For more information contact Dr. Watson at 

watson.leela@ensco.com or 321-853-8264. 
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AMU ACTIVITIES 

AMU Chiefs Technical Activities 
(Dr. Merceret) 

Dr. Merceret and Dr. Willett completed editing the 
history of the Lightning Advisory Panel and the Lightning 
Launch Commit Criteria. It is now available to the public 
(Merceret and Willett, 2010). Dr. Merceret continued 
contributing as an author to the companion Rationale 
document and was also appointed as a co-editor of that 
document. The internal, Government-only version of the 
Rationale was completed except for final editing. 

AMU OPERATIONS (AMU Team) 

IT 

Dr. Bauman and Mr. Barrett completed the annual 
testing of the Contingency and System Security Plans. 

Conferences, Meetings, and Training: 

The following posters and presentation were com- 
pleted for the for the 35 th National Weather Association 
Annual Meeting in Tucson, Ariz., 2-7 Oct 2010: 

• Mr. Barrett prepared a poster titled “Tool For Fore- 
casting Cool-Season Peak Winds Across Kennedy 
Space Center and Cape Canaveral Air Force Sta- 
tion, Phase II”. 
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• Ms. Crawford prepared a poster titled “Modifications 
to the Objective Lightning Probability Forecast Tool 
at Kennedy Space Center / Cape Canaveral Air 
Force Station, Florida”. 

• Dr. Bauman prepared a presentation titled 
“Statistical Analysis of Model Data for Operational 
Space Launch Weather Support at Kennedy Space 
Center and Cape Canaveral Air Force Station”. 

Mr. Roeder of the 45 WS will present both posters 
and the oral presentation on behalf of the AMU. 

Launch Support 

Ms. Crawford and Dr. Merceret supported the Atlas 
V launch on 14 August. 

Personnel Changes 

Due to a reduction in AMU funding in FY11, the 
AMU staff was reduced from 5 full time equivalents 
(FTE) to 4 FTE. Mr. Barrett was reassigned to a new 
position at ENSCO and his last work day in the AMU 
was 1 October 2010. 


LIST OF ACRONYMS 


1 4 WS 1 4th Weather Squadron 

30 SW 30th Space Wing 

30 WS 30th Weather Squadron 

45 RMS 45th Range Management Squadron 

45 OG 45th Operations Group 

45 SW 45th Space Wing 

45 SW/SE 45th Space Wing/Range Safety 

45 WS 45th Weather Squadron 

ADAS ARPS Data Analysis System 

AFSPC Air Force Space Command 

AFWA Air Force Weather Agency 

AMU Applied Meteorology Unit 

ARPS Advanced Regional Prediction System 

CCAFS Cape Canaveral Air Force Station 

CSR Computer Sciences Raytheon 

FSU Florida State University 

FY Fiscal Year 

GSD Global Systems Division 

GUI Graphical User Interface 

JSC Johnson Space Center 
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KSC Kennedy Space Center 

LAP Lightning Advisory Panel 

LDIS Local Data Integration System 

LWO Launch Weather Officer 

MADIS Meteorological Assimilation Data Ingest 

System 

MesoNAM 12-km resolution NAM 

MIDDS Meteorological Interactive Data Display 

System 

MSFC Marshall Space Flight Center 

NAM North American Mesoscale Model 

NOAA National Oceanic and Atmospheric 

Administration 

NWS MLB National Weather Service in Melbourne, FL 

POR Period of Record 

SMC Space and Missile Center 

SMG Spaceflight Meteorology Group 

USAF United States Air Force 

UTC Universal Coordinated Time 

WMO World Meteorological Organization 

WRF Weather Research and Forecasting Model 


The AMU has been in operation since September 1991. Tasking is 
determined annually with reviews at least semi-annually. 

AMU Quarterly Reports are available on the Internet at http://science.ksc.nasa.gov/amu/ . 

They are also available in electronic format via email. If you would like to be added to the email distribution list, 
please contact Ms. Winifred Crawford (321-853-8130, crawford.winnie@ensco.com) . 

If your mailing information changes or if you would like to be removed from the distribution list, please notify 
Ms. Crawford or Dr. Francis Merceret (321-867-0818, Francis. J.Merceret@nasa. gov) . 
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